GSoC 2017 - Coding Period | Week 11

Throughout this week, I was primarily working on the Avro Importer-Exporter modules and improving the documentation of the repository. I also worked on adding the "Export as {format}" button examples for more formats such as JSON & RDS.

On a sidenote, I also attended online test and 3 rounds of interview for Goldman Sachs for the role of Summer Analyst. The online test consisted of 3 sections - Coding, Quant and Machine Learning. I was able to clear the online tests. All 3 rounds of interview were about coding and very light HR questions. However, it turned out to be one of the interviews that go well, but you still don't get selected after the final round and question yourself for a week regarding why you were rejected.

At the beginning of this week, this Pull Request was submitted to add the Avro Importer & Exporter modules. For the Avro IO modules, avro gem has been used. A small code snippet that was used in the Avro Importer, has been added in the previous blog post. Meanwhile, here's a code snippet which makes the Avro Exporter work.

  
  class Daru::IO::Exporters::Avro < Daru::IO::Exporters::Base
    def call
      @schema_obj = process_schema
      @writer     = ::Avro::IO::DatumWriter.new(@schema_obj)
      @buffer     = StringIO.new
      @writer     = ::Avro::DataFile::Writer.new(@buffer, @writer, @schema_obj)
      @dataframe.each_row { |row| @writer << row.to_h }
      @writer.close

      File.open(@path, 'w') { |file| file.write(@buffer.string) }
    end

    private

    def process_schema
      case @schema
      when ::Avro::Schema then @schema
      when String         then ::Avro::Schema.parse(@schema)
      when Hash           then ::Avro::Schema.parse(@schema.to_json)
      else raise ArgumentError, 'Invalid Avro Schema provided.'
      end
    end
  end
  

The Rails App that serves as an example to demonstrate the working of daru-io and daru-view with daru is under active development, in this repository. Support for "Export as {format}" has been added for more formats such as JSON & RDS, in this Pull Request.

Here are a few screenshots depicting the "Export as {format}" buttons and the downloaded files -

Irrespective of how well written the code is, the software won't be friendly for users unless it is well documented. In GitHub, there are quite a few traditional markdown files that are required for any good open-source repository : README, LICENSE, CODE OF CONDUCT, CONTRIBUTING and so on. They are being added quite exhaustively in this Pull Request, to ensure that users and contributors of daru-io, find it easier to develop with daru-io.